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Streaming by 


HLS & MPEG-DASH 
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HLS and MPEG-DASH are the two most widely 
accepted formats. 


The content is divided to small chunks, each 
containing a few seconds of video/audio. 


A playlist file provides the list of all chunks. 


Playlist and chunks can either be pre-processed, 
or be prepared upon request. 
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Pre-packaging to HLS using ffmpeg 


ffmpeg -i video.mp4 -codec copy 


-vbsf h264 mp4toannexb video.m3u8 
-f segment Nx NO 
m BE E F:10, 
Segment lər video.m3u8 vides0000.ts 
-segment time 10 #EXTINF:10 
-segment format mpeg ts video0001.ts 
-segment list type m3u8 #EXTINF:10, 
video$04d.ts video0002.ts 
#EXTINF:10, 
video0003.ts 
MP4 N 
ts is ts 
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Pre-packaging to MPEG-DASH using gpac (MP4Box) 


MP4Box -dash 10000 -rap video.mpd 

-out video.mpd 

-segment-name video <SegmentURL media="video 1.m4s"/> 

video.mp4 ag <SegmentURL media="video 2.m4s"/> 

` <SegmentURL media="video 3.m4s"/> 

<SegmentURL media="vid o 4.m s"/> 

<SegmentURL media="video 5.m4s"/> 

RL de 


«SegmentUl 


media-"video 6.m4s" /> 


MP4 ka 


.m4s .m4s .m4s 
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Turn NGINX 
to a streaming 
server 
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Download and install NGINX RTMP module 


* Developed by Roman Arutyunyan 
* Git: https://github.com/arut/nginx-rtmp-module 
* Blog: http://nginx-rtmp.blogspot.com/ 


sudo apt-get install -y unzip build-essential libpcre3 libpcre3-dev libssl-dev 
cd /opt 
wget http://nginx.org/download/nginx-1.10.1.tar.gz 

wget https://github.com/arut/nginx-rtmp-module/archive/master.zip 
tar -zxvf nginx-1.10.1.tar.gz 

unzip -o master.zip 

cd nginx-1.10.1 


./configure --with-http ssl module --add-module-../nginx-rtmp-module-master 
make 
make install 
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HLS Live 


* Alive application can have HLS mode on. 


e HLS playlist (.m3u8) and segment (.ts) files are generated in real time from 
ingested stream. 


rtmp ( 
server { 
application mylive { 
live on; 
hls on; 


hls path /üsr/local/nginx/html/hls; 
hls fragment 10s; 
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DASH Live 


* Alive application can have DASH mode on. 


* DASH playlist (.mpd) and segment (.m4v and .m4a) files are generated in real-time 
from ingested stream. 


rtmp { 
server { 
application mylive { 
live on; 
hls on; 
hls path /usr/local/nginx/html/hls; 
hls fragment 10s; 


dash on; 
dash path /usr/local/nginx/html/dash; 
dash fragment 10s; 
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VOD streaming by NGINX 


e hls directive turns on dynamic 


HLS segmentation. 
* NGINX Plus can prepare HLS 


manifest (.m3u8) and segment 


(.ts) files on the fly. 


http { 
server { 


application myhls { 


hl 


LS; 


h 
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ls fragment 10s; 


Plus 


f4f directive turns on HDS mode. 


NGINX Plus can prepare HDS manifest 
(.f4m) and segments. 


But, mp4 file needs to be pre-processed to 
f4f using Adobe’s f4fpackager. 


Not much benefit if compared with pre- 
packaging. 


http { 
server { 
application myhds { 
f4f; 
} 
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HLS/DASH 
VOD by 
NGINX 

- How? 
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How to segment 


ffmpeg -ss O -itse b 
-i video.mp4 |-t |10 
-codec cop} 

PI |P|P|P 
segmentl.mp 


20s 
ee _ 20s 30s 40s 
ffmpeg -ss 10 -itbofifset [0 
-i video.mp4 T B PIPIP ffmpeg -ss 20 -itsoffset 20 
-codec copy -i video.mp4 -t 30 
10s segment2.mp4 -codec copy 


segment3.mp4 


30s 
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In reality… 


30s 
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Then, how to cut real mp4? 


-] video. 


segmentl.mp 


" ffmpeg -ss 17 —-itsolffget| 17 


17s 30s 
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The art of getting segment boundaries 


17s 24s 30s 43s 


ffprobe -select streams v -skip frame nokey 
-show frames -v quiet video.mp4 


| grep '^pkt pts time” 2007 
| sed 's/pkt pts time-//' 24.000000 
30.000000 


43.000000 
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Generate HLS playlist 


e The playlist can be generated from a template using key frame locations. 


video.mp4.m3u8 
#EXTM3U 
Segment #EXT-X-VERSION: 3 
: #EXT-X-MEDIA-SEQUENCE: 0 
Template boundaries #EXT-X-ALLOW-CACHE: YES 
Bonn $N=( #EXT-X-TARGETDURATION:10 
HEXT-X-VERS ONS 0.000000, #EXTINF:17.000000, 
= | : 17.000000, video.mp4.0.000000.ts 
#EXT-X-MEDIA-SEQUENCE : 0 P 
= i i 24.000000, #EXTINF:7.000000 
#EXT-X-ALLOW-CACHE: YES / 
#EXT-X-TARGETDURATION: sizeof (SN)  fİ”İ? 2777777 
7 = m. 43.000000, #EXTINF:6.000000, 
i .. video.mp4.24.000000.ts 
FOR EACH UNI 4 müz ) = #EXTINF:13.000000, 
#EXTINF: (SN[i+1] - SN[i]), FISSO 000 eee SES 
$file. $N[i].ts . #EXTINF:8.000000, 
File name video.mp4.43.000000.ts 
} $file= m” 


video.mp4 
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Generate HLS segment 


da 


VN Aa 17 =itsoffset 17 video.mp4.17000.ts 
-i video.mp4 -t 24 I 
-codec copy 
segment2.mp4 


ffmpeg -i segment2.mp4 -codec copy 
-vbsf h264 mp4toannexb 
-flags -global header 
-segment format mpegts 
segment2.ts 
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DASH playlist and segment 


video.mp4.mpd 


<MPD type="static" xmlns="urn:mpeg:DASH:schema:MPD:2011” 


profiles="urn:mpeg:dash:profile:isoff-on-demand:2011” 
Segment minBufferTime-"PT1.5S” 
2 mediaPresentationDuration="PTOHOM40S"> 
boundaries <Period start="PTOS" duration="PTOHOM40S"> 
= <AdaptationSet segmentAlignment="true" bitstreamSwitching="true"> 
$N={ 57 
0.000000 xRepresentation id="avc1:9938805” 
z , 1 — 4 " =" " 
mimeType="video/mp4" codecs-"avcl.42c01e 
17.000000, width="640" height-”264” 
24.000000, startWithSAP="1" bandwidth="144000"> 
<SegmentTemplate presentationTimeOffset="0" timescale="1000” 
initialization="video.mp4.m4v" media="video.mp4+$Time$.m4v"> 
<SegmentTimeline> 
«S t="0" d-"17000"/» 
| «S t="17000" d="7000"/> 
«S t="24000" d="6000"/> 
«S t="30000" d="13000"/> 


30.000000, 
43.000000, 


</SegmentTimeline> 
</SegmentTemplate> 
</Representation> 
</AdaptationSet> 


Template 
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Track split and repackage (MP4Box) 


* Most DASH players don't like video and audio tracks combined in a single 


segment. 


ffmpeg -itsoffset 17 
-i segment2.mp4 
-vcodec copy -an 
segment2.video.mp4 


— MP 
Ly 


4 


MP4 


ffmpeg -itsoffset 17 
-i segment2.mp4 
-vn -acodec copy 
segment2.audio.mp4 


MP4 
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MP4Box -noprog -quiet 
-dash 10000 
-single-segment -rap 
-out video temp.mpd 
-segment-name segment. 
segment2.m4v 


MP4Box -noprog -quiet 
-dash 10000 
-single-segment -rap 
-out video temp.mpd 
-segment-name segment 
segment2.m4v 
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Supporting 
HLS 7 and 
up 
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HLS 7 revision 20 — What's new? 


e HLS now support segments in fMP4. 
* Can share segments with MPEG-DASH and Microsoft Smooth Streaming. 
* Can split video and audio (multiple audio tracks are now cheaper than before). 
e Can save CPU and storage! 


* Master playlist is now mandatory, even if the stream is a single-bitrate. 
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Playlist format ams 


#EXT-X-STREAM-INF : PROGRAM-ID=1 , BANDWIDTH=10130832,CODEC="avcl.640033,mp4a. 
40.2", RESOLUTION=3840x2160 
vide.mp4_1.m3u8 


#EXT-X-MEDIA : TYPE=CLOSED-CAPTIONS, GROUP- 
ID="cc1",LANGUAGE="en",NAME="English", AUTOSELECT=YES, DEFAULT=YES, INSTREAM-ID="CC1" 


#EXTM3U 

#EXT-X-VERSION:3 
#EXT-X-MEDIA-SEQUENCE:1 
#EXT-X-PLAYLIST-TYPE:VOD 
#EXT-X-ALLOW-CACHE : NO 
#EXT-X-TARGETDURATION: 12 


#EXTM3U 

#EXT-X-VERSION: 7 

HEXT-X-MEDIA-SEQUENCE : 1 

#EXT-X-PLAYLIST-TYPE:VOD 
HLS 7 #EXT-X-INDEPENDENT-SEGMENTS 

#EXT-X-MAP:URI="1.mp4" 


7: cal i Master playlist ZEXT-X-TARGETDURATION:12 
#EXTINF:9.801, ee 


vide.mp4+10677.ts 
#EXTINF:9.551, 
vide.mp4+20478.ts 
#EXTINF:11.261, 
vide.mp4+30030.ts 
#EXTINF:10.344, 


#EXTINF:9.80146, 
1+512512.mp4 
#EXTINF:9.55121, 
1+982982.mp4 
#EXTINF:11.26125, 
1+1441440.mp4 


7:77 HLS 7 #EXTINF:10.34367, 
vide.mp4+51634.ts Media playlist 1+1981980 .mp4 


#EXTINF:8.42508, 
1+2478476 .mp4 
#EXT-X-ENDLIST 
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#EXT-X-ENDLIST 


Segments 


Fragmented 


MP4 
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How to support HLS 7 r20 using NGINX RTMP is; 


Segments 


ı #EXTM3U 


| #EXT-X-STREAM-INF : PROGRAM-ID=1, BANDWIDTH=10130832,CODEC="avc1.640033,mp4a. 
| 40.2", RESOLUTION-3840x2160 
| Vide.mp4 1.m3u8 


| #EXT-X-MEDIA: TYPE-CLOSED-CAPTIONS , GROUP- 
| ID="cc1",LANGUAGE="xx",NAME="Undef ined" , AUTOSELECT=YES , Di 
| #EXTM3U rules 


| #EXT-X-VERSION: 7 


#EXT-X-MEDIA-SEQUENCE: 1 
HLS 7 | #EXT-X-PLAYLIST-TYPE: VOD 
Master playlist , #EXT-X-INDEPENDENT-SEGMENTS 


| #EXT-X-MAP: URI="1.mp4" 

' #EXT-X-TARGETDURATION: 12 
| #EXTINF:10.67733, 
1+0.mp4 
| KEXTINF:9.80146, Convert from 
1+512512.mp4 


| #EXTINF:9.55121, DASH mpd 
HLS 7 | 1+982982.mp4 


à : | SEXTINF:11.26125, 
Media playlist | 141441440 .mp4 


| #EXTINF:10.34367, 
141981980.mp4 


| #EXTINF:8.42508, Reuse DASH 
|, 1+2478476.mp4 segments 
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Encryptin 
g HLS 
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Protecting HLS — How does it work? 


== ED 
— oOo 
———— _ 


Decrypt using 
the key 
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Protecting HLS — How to make it work? 


#EXTM3U 

#EXT-X-VERSION:7 
#EXT-X-KEY:METHOD=AES-128,URI="video.key",IV=0x00000000000000000000000000000000 
#EXT-X-MEDIA-SEQUENCE:1 
#EXT-X-PLAYLIST-TYPE:VOD 
#EXT-X-INDEPENDENT-SEGMENTS 
#EXT-X-MAP : URI="1.mp4" 
#EXT-X-TARGETDURATION: 12 
#EXTINF:10.67733, 

1+0.mp4 

#EXTINF:9.80146, 
1+512512.mp4 
ZEXTINF:9.55121, 
1+982982.mp4 


openssl enc -aes-128-cbc -in ”original.mp4" -K ”video.key" -p -iv 0 
-out "encrypted.mp4" 
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4K/8K UHD — How big are they? 


When horizontal and vertical resolutions are doubled, the amount of 
data are quadrupled. As a result, the number of pixels are growing 
exponentially in bigger resolution. 

* SD 640x480 = 300K pixels. 

* HD 1280x720 = 900K pixels (3x SD) 

* UHD 4096x2160 = 8.44M pixels (9.4x HD, 28x SD) 
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Vimeo recommendation 


https://vimeo.com/help/compression 


Quality Bitrate (Mbps) 
so 


x 
ix 
T 
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Optimization for smooth 4K/8K experience 
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Segment duration 
Key frame interval 
Multi-bitrate 

Network optimization 
HEVC/H.265 


2-4 seconds is recommended. 
Longer segments require longer 
' second 
vo is 

RE, mi auam fa aammaad ` is 

(Roughly) half-sized data for the 

same quality, or (also roughly) 

twice quality from the same 

bitrate. 
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Low-latency 
live 
streaming by 
NGINX 
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Low latency live — Why is it a such big challenge? 


HTTP-based streaming was originally not designed for live streaming. 


Total 45-60 | 
seconds? 
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Optimization for sub-10 seconds latency 


* Smaller segment duration (2-3 seconds) 
+ Shorter key frame interval 
* Smaller playlist window (3x segment) 


* Network optimization The smaller, the better, but 
* Chunk Transfer Encoding 5271107 sag Small (Too 
lake the 


Player buffer is beyond ake up after 
“ontrol. We can enforce isturbance.) 
Segments start being to be small, by a small 
sent before they are laylist window. 
complete. 
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Thank you. 
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